feat: multi-provider LLM support via req_llm#22
Merged
Conversation
Replace the monolithic RLM.LLM with a behaviour + two implementations: - RLM.LLM.ReqLLM (new default) — delegates to req_llm v1.6, supports Anthropic, OpenAI, Ollama, Gemini, Groq via "provider:model" specs - RLM.LLM.Anthropic — preserved legacy hand-rolled Anthropic client Add a named model map (config.models) with Config.resolve_model/2 and Config.context_window_for/2. Workers use model_key atoms (:large, :small, or custom) instead of inline config.model_large/model_small lookups. API key resolution now checks ANTHROPIC_API_KEY first, falls back to CLAUDE_API_KEY. All 162 tests pass unchanged (MockLLM unaffected). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix usage key mismatch: check both provider-specific and normalized
key names for cache tokens (cache_creation_input_tokens vs
cache_creation_tokens, cache_read_input_tokens vs cached_tokens)
- Add catch-all clause for unsupported message roles in build_context
- Make resolve_model!/2 actually raise on unknown keys (matching the !
convention) instead of silently falling back to :large
- Add missing {:ok, non_string} clause in Config.resolve_model/2 to
return a descriptive error instead of CaseClauseError
- Fix FallbackLLM default from RLM.LLM (now behaviour-only) to
RLM.LLM.ReqLLM
- Strip "anthropic:" provider prefix in RLM.LLM.Anthropic before
sending to the Anthropic API (models map stores prefixed specs)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Review agents identified several issues across 5 categories:
Silent failures:
- encode_object/1 returns {:error, :no_content} instead of empty string
- Tape.get_events/1 catches :noproc specifically, logs unexpected exits
- FallbackLLM logs when transitioning from tape to live LLM
- extract_usage/1 warns when all usage fields are nil
- context_window_for/2 warns on unknown model keys
Documentation:
- CLAUDE.md: remove stale cost config rows, fix env var references,
fix models default to bare names, rewrite agent orientation section
with key contracts, DI patterns, and common modification patterns
- Fix moduledocs: Replay (default module), Anthropic (differentiator),
Worker (provider-agnostic), LLM behaviour (bare model names)
- Fix "Ollama (via vLLM)" → "Ollama (local models)" in config/req_llm
Examples:
- Update all examples from CLAUDE_API_KEY to ANTHROPIC_API_KEY
- Add examples/local_models.exs for Ollama/local model usage
- Update mix rlm.examples task with local_models entry
Tests:
- Add test/rlm/config_test.exs with 16 tests covering resolve_model/2,
context_window_for/2, load/1 defaults, models map, and API key
Config:
- Remove unused top-level `require Logger` from config.ex
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
errantsky
force-pushed
the
claude/hopeful-haibt
branch
from
11d9cd1 to
ce8a6e8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
req_llmv1.6 — supports Anthropic, OpenAI, Ollama (local models), Gemini, Groq, and any provider req_llm supportsconfig.models) replaces hardcodedmodel_large/model_smallwith flexible%{large: "provider:model", small: "provider:model"}specsRLM.LLM.Anthropicfor users needing direct API controlRLM.Replay.replay/2for deterministic replay of recorded runs, including code patching andfallback: :livemodeChanges
New Modules
RLM.LLM.ReqLLM— default multi-provider LLM backendRLM.LLM.Anthropic— legacy direct Anthropic API clientRLM.Replay— replay orchestrator with patch and fallback supportRLM.Replay.Tape— tape struct + builder from EventLog eventsRLM.Replay.LLM— tape-only LLM behaviour implRLM.Replay.FallbackLLM— tape-then-live LLM behaviour implModified Modules
RLM.LLM— refactored to behaviour + shared utilities onlyRLM.Config— addedmodelsmap,resolve_model/2,context_window_for/2RLM.Worker— uses named model map viamodel_keyinstead of inline lookupsRLM— updated public API to passmodel_key:in worker optsDocumentation & Examples
CLAUDE.md— updated module map, config fields, agent orientation sectionCHANGELOG.md— full Added/Changed/Fixed entriesREADME.md— updated usage examples and env var referencesexamples/local_models.exs— new example for Ollama/local model usageANTHROPIC_API_KEYTests
test/rlm/config_test.exs— 16 tests for Config resolution functionstest/rlm/replay_test.exs— comprehensive replay system testsTest plan
mix compile --warnings-as-errorspassesmix test— 178 tests, 0 failuresmix format --check-formattedpassesRLM.run("Hello", "Echo this", models: %{large: "ollama:qwen3:8b"})mix test --include live_api(requires ANTHROPIC_API_KEY)🤖 Generated with Claude Code